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A WEARABLE COMPUTER IN A 
PROCESS CONTROL ENVIRONMENT 

FIELD OF THE INVENTION 

The present invention relates generally to process control systems and, more 

5 particularly, to the use of a wearable computer to provide enhanced support within 

a process control environment. 

DESCRIPTION OF THE RELATED ART 

Process control systems, like those used in chemical, petroleum or other 

processes, generally include a centralized process controller that is communicatively 

10 coupled to at least one host or operator workstation and to one or more field devices 

via analog, digital or combined analog/digital buses. The field devices, which may 

be, for example valves, valve positioners, switches, sensors (e.g., temperature, 

pressure and flow rate sensors), etc., perform control functions within the process 

such as opening or closing valves and taking measurements of process parameters. 

15 Generally speaking, the process controller receives signals indicative of process 

measurements made by the field devices and/or other information pertaining to the 

field devices, uses this information to implement a control routine and then 

generates control signals which are sent over the buses to the field devices to 

control the operation of the process. Information from the field devices and the 

20 controller is typically made available to one or more applications that are executed 

by the operator workstation to enable an operator to perform any desired function 

with respect to the process, such as viewing the current state of the process, 

modifying the operation of the process, etc. 

While an operator or a technician can access a variety of types of 

25 information pertaining to the process control system and the individual devices 

therein (such as help, diagnostic, set-up and configuration information) using the 

host workstation, there are many process control activities that require a technician 

to go out into the actual plant environment where no host workstation is present. 

Such activities include, for example, visually inspecting a process control device or 

30 area, connecting devices or buses within the process control environment, taking 

manual measurements, repairing and replacing field devices, etc. In these cases. 
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the operator or technician may carry manuals pertaining to the fimction to be 
performed out into the plant and look up any needed information in the field. This 
process can be very cumbersome. More likely, the technician will return to the 
operator workstation one or more times to look up any information he or she may 
5 need during the course of the performing the desired activity, which is very time 
consuming and is prone to error. Other times, the technician will carry a radio or 
walkie-talkie into the plant and communicate via the radio with an operator located 
at the operator workstation to get any needed information. However, the amount of 
information that can be provided over the radio is limited and, again, is prone to 

10 errors because it is based on human communications. Furthermore, because the 

technician typically carries and operates the radio using his or her hands, the use of 
a radio makes performing certain functions, like repairing a device, much more 
cumbersome and difficult. 

With the advent of smaller electronics, portable computers in the form of 

15 wearable computers have become more readily available. A wearable computer 

generally includes a standard central processing unit (CPU) and a memory packaged 
in a small container and placed within a pouch on a belt or harness worn by a user 
(also referred to herein as a "wearer"). Batteries for powering the wearable 
computer are typically located in a different pouch within the harness, which is 

20 designed to make carrying the wearable computer as convenient as possible. 

Peripheral devices, such as disk drives, hard drives, PCMCIA slots, microphones, 
bar code readers and keyboard devices may be communicatively coupled to the 
CPU via appropriate wires or buses and, if desired, one or more of these peripheral 
devices may be placed in or connected to the hamess. It has also been suggested to 

25 provide a heads up display (HUD) worn by the wearable computer user to present 
the wearer with a visual interface. A wearable computer thereby provides portable 
computing power and memory to a user and, because the wearable computer is 
worn instead of carried by the user, the user's hands are only required to 
manipulate a keyboard or other input device. 
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While it has been previously suggested to use wearable computers in 
environments such as office environments, it is not believed that a wearable 
computer has been incorporated in and used in a process control system to enhance 
the abilities of an operator or a technician to identify devices and to perform other 
5 functions within a process control environment. Also, most wearable computers 
require the use of some sort of hand-manipulated input device, such as a keyboard 
or a twiddler. While these devices are typically ergonomically designed to be as 
least cumbersome as possible, these devices still require the use of a the wearer's 
hands to input information or data. In a process control environment however, a 

10 technician typically needs to have both hands free in order to perform complex 

operations, such as calibrating and repairing devices, connecting devices within the 
process control system, etc. 

SUMMARY OF THE INVENTION 
A wearable computer for use in a process control environment includes a 

15 central processing unit and a memory connected to one or more peripheral devices 
including, for example, a heads up display, a microphone, an imaging device (such 
as a video camera) and a remote communication device (such as a wireless ethemet 
transceiver) that conmiunicates with a host computer of a process control system. 
The wearable computer may provide information pertaining to one or more devices 

20 within the process control system via the heads up display. The information which 
can be, for example, diagnostic information, help, operator overviews, schematics 
or process parameter information, may be stored in the memory of the wearable 
computer or may be obtained from the host computer via the remote communication 
device. 

25 The wearable computer may include a software routine or a 

software/hardware device that processes an image developed by the imaging device 
to automatically identify process control devices within the field of view of the 
wearer based on device features. This processing, which can performed on the 
basis of device tags required to be placed on devices within process control 

30 environments, automatically identifies one or more process control devices without 
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requiring the user to input any information via a hand manipulatable device and may 
be used to provide the wearer with information pertaining to the identified devices, 
including process parameter values developed by the identified devices. 

Still further, the wearable computer can be used to test the proper 
5 connection of devices and/or communication channels withm the process control 
system. In this embodiment, a software routine run on the wearable computer 
displays information to the wearer via the HUD including, for example, a list of 
devices or communication channels, and allows the wearer to select an appropriate 
device and/or communication channel to be tested. The wearer may choose a 

10 device or channel using verbal commands decoded by a voice recognition routine 
run on the wearable computer or using commands entered via any other input 
device. After the appropriate I/O channel has been chosen, the routine obtains the 
current value of the signal on the selected channel via remote communications with 
the host computer and displays this value to the wearer on the HUD. At this point, 

15 the wearer can manually test the value on the actual communication channel using, 
for example, a hand-held measurement device. The routine then allows the wearer 
to change the value of the channel by inputting a new value using, for example, 
voice commands. Next, the routine communicates the new value to the host 
computer, which changes that value within the process control system and 

20 conmiunicates the change back to the wearable computer. The change may be 

displayed to the wearer via the HUD, at which point the wearer may again manually 
measure the signal on the channel to see if the measured signal has changed to the 
new value. If not, a problem exists within the process control system configuration. 
Using this system, the wearer can test the connections within a process control 

25 environment in a hands free manner and without having to communicate changes to 
be made to another person located at a different part of the plant (e.g., at an 
operator workstation). 

In another embodiment, routines run on the wearable computer and a host 
workstation enable the wearer and an operator at the host workstation to view and 

30 manipulate a common image to thereby enhance conmiunications between the 
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operator and the wearer. The host system may receive a video signal developed by 
the video camera of the wearable computer and may select a base image for shared 
viewing. This image is displayed on the host display and is sent to the wearable 
computer to be displayed on the HUD. Thereafter, one or both of the operator and 
5 the wearer may manipulate the image by, for example, moving a cursor within the 
image, highlighting, placing data or information on the image, etc. and such 
changes are sent to the other system so as to be displayed on both the host display 
and the HUD. 

In a still further embodiment, the wearable computer may be used to create 
10 and store information, for example, in the form of voice messages, pertaining to 
any device or other object within the process control environment. Such 
information may then be automatically supplied to any wearer or operator who later 
passes by the device. 

BRIEF DESCRIPTION OF THE DRAWINGS 
15 Fig. 1 is a schematic block diagram of a process control network having a 

wearable computer system remotely coupled thereto; 

Fig. 2 is a schematic block diagram of the wearable computer system of Fig. 

1; 

Fig. 3 is a flow chart diagram of a software routine that processes voice data 
20 to recognize commands in the wearable computer system of Fig. 2; 

Fig. 4 is a flow chart diagram of a software routine that automatically 
recognizes process control devices based on video information collected by the 
wearable computer system of Fig. 2; 

Fig. 5 is a flow chart diagram of a set of software routines that provide a 
25 shared image between a host system and the wearable computer system of Fig. 2; 

Fig. 6 is a flow chart diagram of a software routine that provides support to 
a wearable computer user who is verifying communication connections within a 
process control environment; 

Fig. 7 is a first wearable computer screen display used in the software 
30 routine of Fig. 6; 
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Fig. 8 is a second wearable computer screen display used in the software 
routine of Fig. 6; and 

Fig. 9 is another wearable computer screen display used in the software 
routine of Fig. 6. 



Referring now to Fig. 1, a process control system 10 includes a process 
controller 12 connected to a host workstation or computer 14 (which may be any 
type of personal computer or workstation) and to field devices 15, 16, 17, 18 and 
19 via input/output (I/O) cards 20 and 22. The controller 12, which can be by way 

10 of example, the Delta V^^ controller sold by Fisher-Rosemount Systems, Inc., may 
be communicatively connected to the host computer 14 via, for example, an 
ethemet connection and may be communicatively connected to the field devices 15- 
19 using hardware and software associated with any desired communication 
protocol, such as the Foundation™ Fieldbus, the HART*, PROFIBUS*, 

15 WORLDFIP*, Device-Net* or CAN protocols, to name a few. As is typical, the 

controller 12 implements a process control routine stored therein and communicates 
with the devices 15-22 and the host computer 14 to control a process in any desired 
manner. The field devices 15-19 may be any type of devices, such as sensors, 
valves, transmitters, positioners, etc. while the I/O cards 20 and 22 may be any 

20 types of I/O devices conforming to any desired communication or controller 
protocol. 

As illustrated in Fig. 1, the host computer 14 is communicatively coupled to 
a wearable computer system 30 through a remote or wireless communication 
device, such as a remote ethemet transceiver 32. Alternatively, the host computer 
25 14 may be coupled to the wearable computer system 30 via a physical line or bus 
having terminals located throughout the process control environment to which the 
wearable computer system 30 can be temporarily connected and disconnected. 

The wearable computer system 30 includes a wearable computer 34 having a 
remote transceiver 36 and a number of peripheral devices attached thereto. In the 
30 preferred embodiment, the wearable computer 34 includes a Pentium class CPU 



5 
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-6- 




06005/35169 

mother board with video, sound, RAM (e.g., 64 Mb) and ROM with a hard drive 
(e.g., 4.3 Gb), all located within a wearable computer harness (not shown). The 
wearable computer 34 may include any number of slots, such as PCMCIA slots, 
one of which can be used to receive the remote transceiver 36 and another of which 
5 may be used to receive a video processing board such as a video frame capture 
board. The peripherals conununicatively coupled to the wearable computer 34 
include an imaging device 38, which may be a video camera, a HUD 40, a speaker 
42 (which may be a headphone speaker or any other type of speaker), a microphone 
44 and a user input device 46 which may be for example, a typical keyboard, a 

10 mouse, a track ball, or a twiddler device having a limited number of easy to use 
keys (such as function keys), the function of which is defined differently for 
different applications. Of course, any other peripheral devices may be used in 
addition or in the alternative. 

While the imaging device 38 is preferably a video camera, it may instead be 

15 any other type of imaging device, such as a digital camera, that is compact and 
easily transported by the wearer in a hands-free manner. Most preferably, the 
video camera 38 or other imaging device is mounted on the HUD 40 or on some 
other device (such as wearable headgear) which points in the direction that the 
wearer is looking. One small and easily mounted video camera that can be used for 

20 this purpose is sold by the Pulnix corporation. This video camera conforms to the 
high definition television (HDTV) standard (i.e., produces an 800 by 600 color 
pixel image frame), has about one quarter of an inch to one half of an inch diameter 
lens and produces a high resolution color image. However, other video cameras 
can be used instead including, for example, video cameras that produce high or low 

25 definition color or black and white (i.e., gray-scale) images. In some instances, a 
low definition video camera (either color or black and white) may be preferable to 
speed up the time needed to process an image in the manner described below. 

The HUD 40 may use an NTSC video format and is preferably a monocular 
HUD such as the Ml HUD sold by Liquide Image Corp. located in Canada. This 

30 HUD provides a quarter VGA (i.e., 320 by 240 pixel) gray-scale image. Of 
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course, HDTV format HUDs (which are currently prohibitively expensive) or other 
color or gray-scale HUDs, either those available now or those developed in the 
future, could be used instead. The speaker 42, the microphone 44 and the input 
device 46 can be any suitable and easily transportable devices preferably mounted 
5 with respect to the wearer in a hands-free manner. In one embodiment, a bone 
microphone may operate as both the microphone 44 and the speaker 42. As is 
known, bone microphones use the bones within the wearer's jaw to detect voice 
signals and/or to produce sound signals at the wearer's ear. 

With the wearable computer system 30 installed, the wearer still has both 

10 hands free to perform other activities, such as repairing devices, taking 

measurements or holding other instruments. Of course, the input device 46 may 
require one or both hands to operate, but is still preferably mounted in a hands-free 
manner with respect to the wearer. 

Referring now to Fig. 2, the wearable computer 34 includes a CPU 50 

15 coupled to a memory 52, which may be any type of memory including, for 

example, a disk drive (such as a hard, magnetic or laser disk storage device), RAM, 
ROM, EEPROM, EPROM, etc. The CPU 50, which can include one or any 
multiple number of processor units (or other hardwired or firmware elements) 
operating independently or in a coordinated manner, executes one or more software 

20 applications (stored in the memory 52 ) using any of the inputs to the wearable 
computer 34, information stored in the memory 52 and/or information provided 
from the host system via the transceiver 36. The CPU 50 also provides outputs to 
the peripheral devices, as well as to the host system via the remote communication 
device, i.e., the transceiver 36. In the embodiment of Fig. 2, the CPU 50 is 

25 illustrated as including a controller 54 which (may be implemented in hardware or 
software) and which executes the operating system associated with the wearable 
computer 34 to recognize different inputs from the peripheral devices and other 
components of the wearable computer 34 and to execute one or more applications. 
The CPU 50 illustrated in Fig. 2 includes or executes a voice recognition unit 56, 

30 an optical character recognition (OCR) unit 60, a speaker driver 62 and a HUD 

-8- 
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driver 64. Furthermore, the CPU 50 is coupled to a video frame grabber 68 which 
may be provided on a separate video processing board. 

The voice recognition unit 56 which may be, for example, the Dragon 
Dictate system sold by Dragon Systems of Boston, Massachusetts, or any other 
5 desired voice recognition unit, is typically implemented in software but may, 

alternatively, be executed on a separate processor board. In any event, the voice 
recognition unit 56 receives speech, voice or other sound signals from the 
microphone 44, performs voice recognition processing thereon and delivers 
commands to the controller 54 based on recognized voice inputs. The voice 

10 recognition unit 56 may perform any desired or known processing on the received 
voice signals to identify certain recognized speech commands or words. During 
this process, the voice recognition unit 56 may compare an identified voice 
command to a list of stored or recognized speech commands (stored in, for 
example, the memory 52) to determine if a valid command is being delivered by the 

15 wearer. If a recognized command has been received, the voice recognition unit 56 
delivers the command to the controller 54 for further processing. Of course, if 
desired, the controller 54 may determine if a voice command is a valid or 
recognized command within the context of the application being run on the 
controller 54 and may notify the user when an unrecognized command is received. 

20 The voice recognition unit 56 may also have learning capabilities, as is known. 

Fig. 3 illustrates a block diagram of a software routine 80 that processes a 
voice signal to identify voice commands and which may be executed by the 
wearable computer system 30 to enable the wearer to enter data or commands 
verbally and, therefore, in a hands-free manner. A block 82 of the routine 80 

25 receives a voice signal from the microphone 44. A block 84 processes the voice 
signal to identify a voice conmiand within the signal using any desired or standard 
voice recognition processing routine, such as that indicated above. A block 86 then 
compares the identified command or input with a set of commands stored in, for 
example, the memory 52, to determine if the command is valid. If a block 68 

30 determines that the voice conmiand is recognized, a block 90 provides the command 



06005/35169 

to the controller 54 to be used by whatever application is expecting such command. 
Thereafter, or if the voice command signal is not recognized as a valid command at 
the block 88, control is remmed to the block 82 which receives and processes 
fiirther voice signals. Of course, if an invalid command has been received, the 
5 routine 80 may display an indication of such to the wearer. 

The video processing unit provided within the wearable computer 34 of Fig. 
2 includes the frame grabber 68 coupled to the OCR unit 60 but could include other 
video or image processing hardware/software as well. The frame grabber 68 may 
be, for example, a Nogatek board sold by the Nogatek Company, while the OCR 

10 unit 60 may be, for example, the Carina real-time OCR package which is sold by 
Adaptive Recognition Hungary, located in Budapest, Hungary. While this 
particular OCR unit has previously been used to identify licence plate numbers on 
vehicles, it is believed that this product or a derivative of this product (having only 
minor modifications thereto) would operate satisfactorily to recognize device 

15 features as described below. Of course, other suitable frame grabber boards and 

OCR packages could be used as well. As illustrated in Fig. 2, the frame grabber 68 
receives an image signal (having multiple image frames therein) from the video 
camera 38 and provides an output frame to the OCR unit 60. (Of course, if the 
imaging device 38 produces a still image, such as that produced by a digital camera, 

20 the frame grabber 68 may be unnecessary.) 

In one embodiment, the OCR unit 60 processes the received image to 
identify device features within the image, and these device features are then used to 
identify one or more devices within the field of view of the video camera 38. For 
example, the OCR unit 60 may look for and recognize predetermined symbols, such 

25 as alpha-numeric symbols located on field devices, and provide such recognized 
symbols to the controller 54 for device identification. Of course, if desired, the 
output of the video camera 38 may be used for other purposes. For example, the 
video image may be provided to the controller 54 to be displayed on the HUD 40 
and/or may be sent to the host computer 14 via the transceiver 36 for viewing or 

30 and/or processing by the host computer 14. 

- 10- 
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Referring to Fig. 4, a routine 100 illustrated in flow chart form may be 
implemented in software executed by the wearable computer 34 to automatically 
recognize devices within the field of view of the wearer based on the video input 
collected by the imaging device 38. A block 102 obtains a video or other image 
5 from the imaging device 38. If the imaging device 38 is a video camera, the block 
102 may use the frame grabber 68 to grab a particular video frame. However, if 
the imaging device is, for example, a digital camera, the block 102 may access the 
image directly without the aid of the frame grabber 68. 

A block 104 then processes the obtained video image or frame to identify 

10 potential device features within the video frame. In one embodiment, the device 
feature is a device tag mandated to be placed on each of the field devices within a 
process control environment by OSHA. Usually, such device tags include a 
rectangular holder or frame (typically one to two inches high by three to four inches 
wide) having alpha-numeric characters etched or otherwise engraved or carved 

15 therein so as to be visible to persons within the process environment. The alpha- 
numeric characters are usually a different color than the frame to make these 
characters more visible. When recognizing device tags, the block 104 scans the 
image to identify areas likely to contain device tags, such as rectangular areas 
within the image, areas with certain ranges of colors, areas having alpha-numeric 

20 characters therein, etc. Of course any desired processing may be used to search for 
these device features. Thereafter, a block 106 recognizes or decodes the device 
features within the identified areas. In particular, when device tags are being 
identified, the block 106 may apply optical character recognition (using the OCR 
60) on the identified features to produce a preliminary device ID. If more than one 

25 device is within the image being processed, the blocks 104 and 106 may recognize 
numerous device features (such as device tags) and identify numerous preliminary 
device IDs. 

Next, a block 108 compares each of the preliminary device IDs to a list of 
device IDs stored in, for example, the memory 52 to verify the existence of devices 
30 corresponding to the preliminary device IDs. If corresponding devices exist, the 
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device IDs are verified and each of the verified IDs is provided by a block 110 to 
the controller 54 for use in other applications, to be displayed to the wearer via the 
HUD 40 and/or to be sent to the host computer 14 via the transceiver 36. 

While the routine 100 can identify devices based on any observable features, 
5 it is preferable that the routine 100 identify devices based on device features, i.e., 
features that are part of the device as it is placed in the field without regard to 
automatic detection and identification by the wearable computer system 30. In 
other words, while it would be possible to place bar codes or other unique 
identifiers on each of the devices within a process control environment, it is 

10 preferable to have the routine 100 identify devices based on features that are not 
placed on the device solely for the purpose of detection by the wearable computer 
system 30, i.e., features already existing on the device for other purposes. If 
detection and identification is performed using device features, then no additional 
steps need to be taken to label or otherwise mark each device within a process 

15 control environment for the specific purpose of being identified by a wearable 
computer. 

Other applications which, for example, automatically display information to 
the wearer via the HUD 40 may display the identified devices to the wearer, may 
display other information pertaining to the identified device(s) to the wearer via the 

20 HUD and/or may send the identified device IDs to the host system 14. Of course, 
the list of recognized devices may be stored in the memory 52 of the wearable 
computer 34 or within a different memory, such as a memory within the host 
system 14 which can be accessed via remote communications by the block 108 to 
verify preliminary device IDS. As will be understood, it is not essential that each 

25 of the blocks of the routine 100 be executed within the wearable computer system 
30. Instead, one or more of these blocks can be executed by the host computer 14, 
which can communicate with the wearable computer system 30 to perform the 
routine 100. 

Referring again to Fig. 2, the speaker driver 62 takes signals provided by 
30 the controller 54 and processes them by, for example, converting them to standard 

- 12 - 
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analog audio signals, amplifying them, etc. The speaker driver 62 then provides 
the processed signal to the speaker 42. As will be readily understood, the speaker 
driver 62 and the controller 54 may be used to play prerecorded signals stored in, 
for example, the memory 52 or the memory of the host computer 14 and/or may be 
5 used to relay real-time audio produced by or at the host system, such as the voice of 
an operator located at the host system, or the voice of another wearable computer 
user located elsewhere within the process control environment. The voice or audio 
signals to be played on the speaker 42 may be provided to the wearable computer 
34 via the transceiver 36 from the host system or may be provided using any other 

10 audio communication system coupled to the wearable computer 34. 

Similarly, the HUD driver 64 receives signals from the controller 54 
including graphics to be displayed on the HUD 40, and preforms appropriate 
processing on these signals for display via the HUD 40. In some embodiments, the 
HUD driver 64 and the HUD 40 may be used in conjunction with the twiddler 46 or 

15 microphone 44 to provide a standard computer operating environment, such as a 
Windows image having dialogue boxes, text, graphics and the like. With this 
environment, the wearer can move a cursor, enter information or manipulate the 
image on the HUD 40 to, for example, run an application or make decisions within 
the context of an application being executed by the wearable computer 34. 

20 The controller 54 uses the transceiver 36 in any desired or standard manner, 

and provides signals to the transceiver 36 for communication to the host system 
using any desired communication protocol. Likewise, the controller 54 receives 
and decodes conununications from the host computer 14 via the transceiver 36 using 
any desired communication protocol. 

25 The wearable computer system 30 of Fig. 2 can be used to provide 

numerous kinds of information to the wearer and/or to perform functions within the 
process control environment which make the wearer's task easier and quicker when 
the wearer is, for example, inspecting, installing, repairing, calibrating and 
checking the connections of different devices within the process control 

30 environment. For example, using the wearable computer system 30, a wearer can 
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obtain and view information pertaining to certain devices or areas within the process 
control environment via the HUD 40 either automatically or after appropriate input 
via one of the peripherals. The wearable computer 34 may store, or may 
communicate with the host computer 14 to obtain, any desired information 
5 pertaining to a particular device or to the process control system in general and 
display that information to the wearer via the HUD 40 at the request of the wearer 
or when the wearable computer system 30 recognizes a device within the field of 
view of the wearer as described above. The displayed information may include 
process information, such as schematics or operator overviews of the process 

10 control system, device information such as device lists, help information, diagnostic 
information and even process parameter information (such as measurements, 
parameter values, etc.) made by or associated with one of more of the devices 
connected within the process control system. 

To view such information, the wearer can, when walking by a device, enter 

15 a device identifier, such as a device tag or a device number, which may cause the 
controller 54 to automatically display certain kinds of device information, such as 
help, calibration, diagnostics, parameter values, etc. Of course the wearer can 
enter the device identifier using the twiddler 46, the microphone 44 or any other 
input device. When using the microphone 44, the voice recognition unit 56 can 

20 identify, for example, a spoken device tag number or name and provide that device 
tag number or name to the controller 54. If desired, the voice recognition unit 56 
can be set up to receive a device number, a device name or any other device 
identifier and compare the entered identifier to a list of valid device numbers or 
names within the memory 52. 

25 In one embodiment, as described above, the devices within the field of view 

of the wearer are automatically detected by the video processing circuitry and, when 
such detection takes place, information about the device may be automatically 
displayed to the wearer via the HUD 40 in any desired format. If the information is 
stored in the memory 52, the information can be automatically accessed by the 

30 controller 54 and provided or displayed via the HUD 40 using the HUD driver 64. 
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Alternatively, if the information is stored within the host system, the controller 54 
may request and receive the appropriate information via the transceiver 36 and then 
display such information on the HUD 40. In the case of process parameters 
measured by or stored within a device, the host system may communicate with the 
5 device to obtain the most recent values or data before delivering that information to 
the wearable computer system 30. 

In any of these cases, the controller 54 can display a list of recognized 
devices to a user and allow the user to choose to view information about any of the 
devices or, alternatively, the controller 54 can automatically display information 

10 about the recognized devices via the HUD 40. Significantly, the use of the 

microphone 44, the video camera 38 and the associated hardware/software on the 
wearable computer system 30 enables the wearer to receive and view information 
pertaining to devices (or areas or other units of the process control system) 
automatically in a hands-free manner, i.e., without having to enter any data or other 

15 information via a hand-held or hand manipulated device. This leaves the wearer's 
hands free to perform other tasks, such as repairing, replacing or calibrating a 
device, manipulating other tools, etc. which is very advantageous. Still further; the 
wearable computer system 30 can receive and display information measured by or 
stored within devices at which the wearer is actually looking, without the need for 

20 separate dials or displays being physically located on the outside of each device. 

In another embodiment, the wearable computer system 30 can be used to 
provide a shared view (e.g., display) to an operator located at, for example, the 
host computer 14 and to the wearer via the HUD 40 to thereby enhance 
communications between the two. Such a shared view application displays the same 

25 image to both persons and allows one or both of these persons to manipulate the 
image to, for example, point out or highlight particular parts of the image or post 
data on the image. These actions can be used in conjunction with voice 
communications to thereby enhance conversations between the wearer and an 
operator located at the host computer 14. 
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Fig. 5 illustrates a block diagram of a software routine 116 that can be run 
on the host computer 14 and a block diagram of a software routine 118 that can be 
run on the wearable computer system 30 to implement a shared view or display. 
The routine 118 includes a block 120 that collects and sends a video image to the 
5 host computer 14 via the transceiver 36. (Communications between the wearable 
computer system 30 and the host computer 14 are illustrated in Fig. 5 by dotted 
lines.) This image may be the entire multi-frame image produced by the video 
camera 38 or may be any one or more individual frames thereof. A block 122 
within the routine 116 receives the video image and a block 124 displays the video 

10 image to the operator via a display device associated with the host computer 14. A 
block 126 enables the operator at the host computer 14 to choose a frame of the 
video image to be used as the basis for the shared view (a base image). The block 
126 may, for example, simply display the most recently received frame of the 
received video signal and wait for the operator to indicate that a freeze of the image 

15 is requested. Alternatively, the block 126 may allow the operator to replay 

received frames to choose a desired image or may allow the operator to choose a 
base image in any other desired manner. If the operator does not choose a base 
image for the shared display, the block 126 provides control back to the block 122. 
If the operator chooses a base image at the block 126, a block 128 sends the 

20 selected base image to the wearable computer system 30 for display to the wearer 
on the HUD 40. The block 128 may also, if desired, display the selected base 
image to the operator via the display of the host computer 14. 

Next, a block 130 within the routine 116 determines whether changes to the 
base image are being made or requested by the host computer operator. Such 

25 changes may include, for example, moving a cursor or a pointer, drawing on the 
image, highlighting areas of the image, posting information or other data on the 
image, or any other desired changes which enable the operator to conmiunicate with 
the wearer using the image. These changes may be made by the operator using any 
desired operating system protocols and peripheral devices, such as a mouse and a 

30 keyboard. If changes to the image are made by the operator, a block 132 sends the 
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changes to the wearable computer system 30 via the transceiver network 32/36. 
The changes may be conraiunicated using any desired protocol and either the 
specific changes being made or an entire new image frame having the changes 
therein can be sent to the wearable computer system 30, as desired. In one 
5 embodiment, changes to the image in the form of pointer movements may be 

communicated as new pointer coordinates. After image changes have been made 
and sent to the wearable computer system 30, or if no new changes are made by the 
host operator, a block 134 refreshes the image of the host system (incorporating 
changes made by the operator as well as changes made by the wearable computer 

10 system and sent to the host system). Control of the routine 118 is then returned to 
the block 130 to detect other changes made by the host operator. 

Meanwhile, the routine 118 includes a block 136 that displays the base 
image received from the host system on the HUD 40. A block 138 then detects 
changes to the image made by the wearer, which changes can be made using any 

15 available input device including the microphone 44 and the twiddler 46. If the 

wearer makes changes to the displayed image, a block 140 sends the changes to the 
host computer 14. Thereafter, or if no wearer initiated changes are detected, a 
block 142 refreshes the image on the HUD 40 incorporating changes made by the 
wearer as well as changes made by and received from the host computer 14. 

20 Control of the routine 118 is then returned to the block 138 for detection of further 
wearer initiated changes. 

In this manner, the routines 116 and 118 operate on the host computer 14 
and on the wearable computer system 30 to provide a shared view or scene that can 
be manipulated by one or both of the host operator and the wearer to enhance 

25 communications between the two. While the base image has been described herein 
as being derived from an image collected by the wearable computer system 30, this 
need not be the case. The base image could, instead, be a stored operator view, 
schematic, etc. related to the process or device of interest. In either case, the 
shared view enables the host operator to point out and talk about different elements 

30 within the displayed image in a manner that is easily viewable by the wearer. 
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Furthermore, if desired, the wearer can make changes to the image using, for 
example, the same or a different cursor to aid conversations with the host operator. 
If desired, the wearer need not be able to make changes to the image, which 
simplifies the routines 116 and 118 of Fig. 5. Also, if desired, the wearer may 
5 select the base image to be used before it is sent to the host computer 14. 

Another use of the wearable computer system 30 within a process control 
environment will be described in conjunction with the routine 150, illustrated in 
flow chart form in Fig. 6, which is preferably executed within the wearable 
computer system 30. Generally speaking, the routine 150 enables the wearer to 

10 check out and verify the proper connection of different devices or communication 

channels (such as I/O connections) within a process control environment in a hands- 
free manner and without the aid of an operator at a host device. Previously, 
verifying the proper connections of the devices or conmiunication channels within a 
process control environment required a technician to go out into the field with a 

15 hand-held measurement device, such as a voltmeter, and a hand-held radio which 
the technician used to communicate with an operator at a host workstation. The 
technician first had to go to a device, indicate to the host operator via the hand-held 
radio that he or she was at the device and then indicate which conmiunication 
channel he or she was going to check. At this point, the technician had to take a 

20 hand-held meter and actually measure the signal on the line. The technician then 

told the host operator, via the hand-held radio, what the measured signal was so that 
the host operator could verify whether the measured signal was the actual signal on 
the selected communication channel. Thereafter, the technician would tell the host 
operator to change the signal on the channel in question and the host operator would 

25 cause the signal or value of the communication channel to be changed. The 

technician would then measure the signal on the channel again to see if the change 
actually occurred. As is evident, this process required a lot of cumbersome 
communications between the host operator and a technician and was difficult to 
implement in a large and complex process control environment where the technician 

30 was trying to simultaneously manipulate a hand-held radio, a hand-held meter and 
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obtain access to appropriate devices or communication lines. Furthermore, this 
process relied on conmiunications between a host operator and the technician which 
tended to create confusion and to introduce errors based on miss-communications. 
Using the routine 150 of Fig. 6, a wearer can check out device 
5 communication channel connections, such as I/O connections, within a process 

control system in a relatively hands-free manner (i.e., holding only a measurement 
device) and without the need to communicate with an operator located at a host 
workstation. Instead, the wearable computer system 30 communicates directly with 
the host computer to provide the wearer with all the information he or she needs 

10 and to make changes requested by the wearer necessary to check out the 

connections of a device or a communication channel within the process control 
system. Using the routine 150, the wearer can go out into the process control 
environment, obtain a list of devices and/or communication channels associated 
with a device, choose a particular device and/or communication channel for testing, 

15 find out what the signal on the device or channel being tested should be, make 

changes to the signal and measure both the original signal and the changed signal to 
test the proper connection of the device or channel, all without the need for a host 
operator. 

The routine 150 includes a block 152 that displays a list of devices that may 
20 be tested on the HUD 40. The wearer may select a particular device to be tested by 
selecting one of the listed devices in any desired manner. Preferably, the wearer 
speaks commands into the microphone, such as UP, DOWN, LEFT, RIGHT, 
ENTER, etc. which are recognized and provided to the controller 54 and are used 
to move a cursor (which may be a highlighted area) or to select items displayed on a 
25 Windows screen on the HUD 40. Of course, the wearer may also select a device 
using the twiddler 46 or other keyboard device, by using the microphone to enter 
the name or tag associated with a device, or using the video camera 38 to 
automatically identify a device as described with respect to the routine 100 of Fig. 
4. 
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A block 154 waits for the wearer to select a device and, after a device is 
selected or otherwise chosen by the wearer, a block 156 displays, via the HUD 40, 
a list of communication channels associated with the selected device. An example 
of such a display using a Windows-type display screen is illustrated in Fig. 7 and 
5 includes a set of 11 communication channels for the device CTLRl (controller 1) 
with the first channel CTLR1C02CH01 being highlighted. Of course, the list of 
I/O or other communication channels may be displayed in any other manner and is 
not limited to that of Fig. 7. 

Referring again to Fig. 6, a block 158 waits for the wearer to select a 

10 communication channel to be checked. The wearer may select a particular channel 
displayed in, for example, the screen of Fig. 7 using simple voice commands such 
as BACK and NEXT to move the cursor to a different channel and ENTER to select 
that channel. Thus, to select the third communication channel (CTLRl C02CH03) 
when viewing the display screen of Fig, 7, the wearer may simply say NEXT twice 

15 to highlight the channel CTLR1C02CH03 and then say ENTER to select that 

channel. While other voice commands can be used, it is preferable to limit the set 
of voice conmiands to simple words that can be recognized more easily by the voice 
recognition unit 56. Also, while the display screen of Fig. 7 may be manipulated 
using other input devices, such as the twiddler 46, it is preferable to enable the 

20 wearer to manipulate the screen and select or enter data on the screen using voice 
signals or using other hands-free input devices which allow the wearer to use both 
of his or her hands for other activities. 

After a user has selected a particular conmiunication channel to check, a 
block 160 displays a further screen on the HUD 40 which indicates process 

25 information corresponding to the selected channel. An example of such a screen is 
illustrated in Fig. 8 for the selected channel of CTLR1C02CH01. To create the 
screen of Fig, 8, the block 160 obtains the current process value of the selected 
communication channel from the host system via the transceiver 36 and displays the 
current value of that channel (in this case "0") along with an indication of the quality 

30 of the signal (in this case "good"). The block 160 also provides an area for the user 
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to enter a new process value for the channel and indicates the type of signal on that 
channel, that is, whether the channel is an analog channel or a digital channel, and 
the valid ranges of that signal. The information displayed on the screen is either 
stored in the memory 52 of the wearable computer system 30 or is obtained from 
5 the host computer 14 which either stores that information in a memory or obtains 
the information from a device. In the illustrated example of Fig. 8, the channel 
CTLR1C02CH01 is a digital channel currently set to the value of zero. Fig. 9 
illustrates a similar screen displayed on the HUD 40 for the channel 
CTLR1C06CH01 which is an analog channel having a valid range of 0-100 and 

10 which has a current value of 90. 

When viewing the screen of Figs. 8 or 9, the user can manually measure the 
value on the selected channel using, for example, a hand-held voltmeter or any 
other device. If the measured value is the same as the value listed in the current 
value field of the screen, then the wearer can continue by entering a new value in 

15 the new value field. Referring again to Fig. 6, a block 162 waits for the wearer to 
enter a new process value, preferably using voice commands in the form of 
numbers and other simple conmiands such as ENTER, BACK and NEXT, so that 
the wearer does not have to remove his or her hands from the metering device. A 
new value of 98.5 is being entered into the new value field of the screen display of 

20 Fig. 9. Upon receiving a new value, a block 164 sends that new value to the host 
system which then changes the selected channel to the new value and, after 
verifying that the selected channel has been changed to the new value, sends the 
new value to the wearable computer system 30 as the current value of the selected 
channel. A block 166 then refreshes the screen display on the HUD 40 to indicate 

25 that the current value has been changed to the previously entered new value and 
clears the new value field to enable the wearer to enter a different new value. At 
this time, the wearer can measure the signal on the selected channel using the hand- 
held meter to see if the signal has changed to the entered new value. If so, then the 
communication channel is most likely correctly connected and operating within the 

30 process control system. If not, then a problem exists which must be identified and 
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corrected. Of course, the wearer may make further changes to the communication 
channel value and measure those changes, or may scroll back to the channel or 
device selection screens to select a different channel or device to be checked. 

Using the system described above, a single person may verify the proper 
5 connection and operation of different communication channels within a process 
control environment without needing to talk to and coordinate with an operator 
located at a host station and without needing to carry around a hand-held radio 
which gets in the way of the measurements and other activities being performed in 
the field. 

10 In another embodiment, the wearable computer system 30 can be used to 

store and automatically retrieve information pertaining to any device or object 
within a process control environment, including devices that have device tags or 
other recognizable device features and objects such as walkways, trash cans, 
buildings, etc. that do not typically have device tags associated therewith. Using 

15 the wearable computer system 30 in this manner, a wearer can walk around a plant 
or other process control environment and record voice messages (or other 
information or data) pertaining to devices or objects within the plant for future 
retrieval either by that wearer or by another person. Likewise, upon seeing a 
device or other object, the wearer can determine (by looking at the display on the 

20 HUD 40) if any voice messages have been previously created for that device and 
can retrieve such previously created voice messages. 

In one embodiment, a software routine for implementing this functionality 
(which may be stored in and executed by the processor or CPU 50 of the wearable 
computer 34) includes three basic routines, which may be separate routines or 

25 which may all be subparts of a single routine. The first routine identifies one or 
more devices within the field of view of the wearer or as being of interest to the 
wearer. This routine may, for example, accept voice inputs (from the microphone 
44) in the form of device names, tags or other device identifiers to identify devices 
that are currently of interest to the wearer. Similarly, this routine may display a list 

30 of devices to the wearer via the HUD 40 and enable the wearer to select one of the 
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displayed devices using, for example, voice commands or other inputs. 
Alternatively, this routine may automatically identify devices using the video image 
processing routine described above with respect to Fig. 4, which identifies one or 
more visible device features. Instead of using device features, the automatic video 
5 processing routine may identify a device based on identifiers placed on or near the 
device for the specific purpose of identifying the device (such as optical bar codes). 
On the other hand, transmitters may be placed on or near one or more devices and 
these transmitters may send out a signal which is received by the wearable computer 
34 and decoded by the routine to identify the one or more devices. In one 

10 embodiment, a single transmitter may be used for a room or other unit area and, 
upon receiving and decoding the transmitted signal, the routine may access a 
memory (located, for example, in either the wearable computer 34 or the host 
computer 14) which stores all of the devices within that room or unit area. A list of 
these devices may then be provided to the wearer via the HUD 40. Similarly, 

15 devices that do not have tags or other automatically recognizable features may be 
tied (in a database) to devices that have such automatically recognizable features. 
Typically, devices in close proximity to one another will be tied together (associated 
with one another) in the database. Thereafter, whenever one of the devices having 
an automatically recognizable feature (a tagged device) is identified, the routine may 

20 consult the database to determine other non-tagged devices that are near to, or that 
are otherwise associated with the tagged device and display a list of all of these 
devices to the wearer via the HUD 40. Of course, other methods of identifying 
devices can be used as well. 

When one or more devices have been identified and, for example, displayed 

25 to the wearer via the HUD 40, a second routine enables the wearer to store a voice 
message to be associated with one of the identified devices. The wearer may select 
one of the identified devices (using, for example, voice commands or any other type 
of input) and then, when prompted via the HUD 40, speak into the microphone 44 
to create a voice message. The second routine then stores the voice message in a 

30 memory as being associated with the identified/selected device. This memory may 
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be the memory 52 on the wearable computer 34 or, preferably, may be a memory 
somewhere within the host system such as in the host computer 14. When stored 
on the host computer 14, the voice message is available to more than one wearable 
computer. 



any of the devices identified by the first routine and, if so, displays in indication, 
such as an icon, on the HUD 40 to tell the wearer that a previously stored message 
exists for that identified device. When the wearer selects the icon using, for 
example, voice commands, the third routine retrieves the previously stored voice 

10 message from the memory and plays it to the wearer via the speaker 42. 

Using this data storage/retrieval unit, whenever a wearer (or an operator of 
the host system 14) identifies a device, either manually or automatically, the wearer 
(or the operator) can record a voice message to be associated with that device and 
can, likewise, retrieve and hear previously stored voice messages associated with 

15 that device. In this manner, a wearer (or operator) may make notes or leave 

messages about a device or other object within the process control system which can 
later be retrieved by the same or a different person. Such a message may, for 
example, inform the next person that repair is ongoing on the device, or that 
calibration of the device needs to be performed, or may be any other desired 

20 message pertaining to the device or object. In one simple example, a wearer may 
walk down a walkway within the process control environment and notice that the 
walkway needs to be repainted or repaired. (The walkway may be identified 
automatically based on the room that the user is in, based on the proximity of the 
walkway to other devices that can be automatically identified using device features, 

25 based on specific codes or other features placed on the walkway to enable automatic 
identification, based on user generated input of any kind including voice input and 
hand operated device input, or in any other manner.) The wearer may select the 
walkway on the HUD 40 and then make a voice message indicating the repair to be 
made to the walkway. Thereafter, whenever the walkway is recognized as being of 

30 interest or as being viewed by a wearer of a wearable computer (or an operator at 



5 



A third routine determines if any previously stored voice messages exist for 
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the host computer 14), the voice message is automatically made available to that 
wearer (or operator) and is indicated as being available by an icon (which may also 
be a text message) associated with that walkway on the HUD 40. In this manner, 
new information may be created and stored as associated with any device or object 
5 within a process control environment and this information may be later provided to 
a user in the same manner and/or at the same time that other, more standard 
information (such as help information) is made available to a user. 

The routines described herein may, of course, be implemented in a standard 
multi-purpose CPU or on specifically designed hardware or firmware as desired. 

10 When implemented in software, the software may be stored in any computer 

readable memory such as on a magnetic disk, a laser disk, or other storage medium, 
in a RAM or ROM of a computer or processor, etc. Likewise, this software may 
be delivered to a user or a device (such as the wearable computer) via any known or 
desired delivery method including, for example, on a computer readable disk or 

15 other transportable computer storage mechanism or over a communication channel 
such as a telephone line, the internet, etc. (which is viewed as being the same as or 
interchangeable with providing such software via a transportable storage medium). 

While the present invention has been described with reference to specific 
examples, which are intended to be illustrative only and not to be limiting of the 

20 invention, it will be apparent to those of ordinary skill in the art that changes, 
additions and/or deletions may be made to the disclosed embodiments without 
departing from the spirit and scope of the invention. 
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